System and method for price testing and optimization

ABSTRACT

A method includes generating a control set, based on input received via a user interface, the control set comprising at least a first product of a product category type. The method further includes generating a test set, based on input received via the user interface, the test set comprising at least a second product of the product category type other than the first product in the control set. The method further includes changing a feature of the second product in the test set, the feature being visible on a web page via the internet, while maintaining the feature of the first product in the control set. The method further includes measuring competitor or consumer responses to the changing. The method further includes generating a recommendation based on the measured competitor or consumer responses.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 13/841,629 (now granted as U.S. Pat. No.10,290,012), filed Mar. 15, 2013, and entitled “SYSTEM AND METHOD FORPRICE TESTING AND OPTIMIZATION,” which is hereby incorporated byreference in its entirety. This application also claims priority to U.S.Provisional Patent Application No. 61/730,801, filed Nov. 28, 2012, andentitled “SYSTEM AND METHOD FOR PRICE TESTING AND OPTIMIZATION,” whichis hereby incorporated by reference in its entirety. This application isalso related to, commonly assigned U.S. patent application Ser. No.13/841,322 (now granted as U.S. Pat. No. 9,928,515), filed Mar. 15, 2013and entitled “SYSTEM AND METHOD FOR COMPETITIVE PRODUCT ASSORTMENT,”U.S. patent application Ser. No. 13/837,644, filed Mar. 15, 2013, andentitled “SYSTEM AND METHOD FOR AUTOMATIC WRAPPER INDUCTION BY APPLYINGFILTERS,” U.S. patent application Ser. No. 13/837,961 (now granted asU.S. Pat. No. 9,223,871), filed Mar. 15, 2013, and entitled “SYSTEM ANDMETHOD FOR AUTOMATIC WRAPPER INDUCTION USING TARGET STRINGS,” and U.S.patent application Ser. No. 13/838,195, filed Mar. 15, 2013, andentitled “SYSTEM AND METHOD FOR AUTOMATIC PRODUCT MATCHING,” thedisclosures of which are incorporated by reference in their entirety.Copies of U.S. patent application Ser. Nos. 13/841,322, 13/837,644,13/837,961, and 13/838,195, are attached hereto as Appendices A, B, C,and D respectively.

TECHNICAL FIELD

This disclosure relates generally to the field of pricing retailproducts and particularly to revenue management, price sensitivityanalysis and price optimization of retail products in an onlineenvironment.

BACKGROUND

Competitive intelligence as it relates to pricing has been an importantaspect of the retail business for decades. Today, via the internetconsumers have tools that allow them to compare prices across thousandsof retailers in seconds.

Many retailers carry a very large number of products on their catalog,often times m excess of 100,000 different stock keeping units (SKUs)associated with different products. Each SKU is often sold by manydifferent competitors at different prices. However, competitors maychange their prices for products at any time, which makes it moredifficult to determine the pricing of the products at differentretailers. Different retailers selling a plurality of products atdifferent prices create a massive amount of information to be analyzedon a timely manner. Because of the massive amount of informationassociated with competitive intelligence, oftentimes retailers findthemselves with product prices that are either too high or too low. Aretailer having sub-optimal product pricing can results in either lowsales or poor margins.

Current price optimization and sensitivity analysis techniques relyprimarily on historical pricing data and consumer facing website clickstream data. These solutions simply react to fluctuating sales volumesand do not take into account how competitors react and respond to pricechanges to a product on a channel, pricing and promotions and freightpricing strategies. To this end, there is a need for improved systemsand methods for revenue management, price sensitivity analysis and priceoptimization utilizing competitive data and web analytics.

SUMMARY

Embodiments disclosed herein include a system that may determine webpages of competitors containing products relevant to a customer of thesystem. One example of such a customer may be a business entity. Oneexample of a business entity can be a retailer. This retailer may beselling a product and is interested in information relating to thatproduct or similar ones from its competitors, including known andunknown competitors. These competitors may have a presence on theInternet. The system may be configured to pull information associatedwith products or product types from an unbound number of domains on theInternet. Examples of information associated with a product may includea product name, associated competitor's name, brand, description,product attributes, SKU, price, image, time, date etc. These competitorsas well as their domains and websites may or may not be known by acustomer requesting the information. The pulled information associatedwith a product may be stored in a data store, and may be included as aninstance in a product table where the pulled information associated witha product from a competitor is arranged in the same row of the producttable.

In this disclosure, the term “domain” is used in the context of thehierarchical Domain Name System (DNS) of the Internet. Those skilled inthe art appreciate that the DNS refers to a hierarchical naming systemfor computers or any resource connected to the Internet. A network thatis registered with the DNS has a domain name under which a collection ofnetwork devices are organized. Today, there are hundreds of millions ofwebsites with domain names and content on them. As the number ofwebsites continues to grow, pulling information associated with aproduct or products from an unbound number of domains on the Internetcan be a very complex, tedious, and complicated process.

Embodiments disclosed herein can leverage wrapper induction and wrapperinfection methodologies disclosed in Appendix A and Appendix B attachedherewith to automate a data mining process across unbounded domains.Additionally, because each competitor may describe or define a productin different ways, it may be desirable or necessary to determine whichproducts sold by different competitors refer to the same product.Embodiments disclosed herein can also leverage a novel approachdisclosed in Appendix C attached herewith to match a product or producttype of interest with product information crawled from the Internet.This matching process can help to ensure that any price or featurecomparison made between a predefined product/product type andproducts/product types being sold by different competitors on theInternet are the same and/or relevant. Appendices A, B, and Care herebyincorporated by reference in their entireties.

Embodiments as described herein relate to price optimization systems andmethods configured to use data of competitor's pricing for products,competitors price responses associated with products based on acustomer's changes to prices of products and consumer navigationbehavior on a channel associated with the customer that is updatedfrequently to derive price sensitivity data. Embodiments may acquirecompetitive data and use this data to improve current price optimizationtechniques. The effect of adding massive and continuously updatedcompetitive datasets result in more predictable consumer web analyticsof a consumer's website over longer periods of time, which may createmore robust optimization models configured to handle long-term trends incustomer loyalty and price optimization across a wider range of productsand/or marketing strategies.

Embodiments described herein may (1) gather data, (2) design tests, (3)run tests, (4) perform sensitivity analysis, and/or (5) create pricerecommendations for products. In one embodiment, a price testing systemand method may gather the following data to optimize pricing of aproduct: (1) how quickly competitors respond to a change in price for aproduct, (2) how aggressively competitors change a price of acorresponding product, (3) number of unique visitors/number of visits,(4) conversion rates (i.e., the percentage of visitors to a product pagewho end up purchasing such a product), (5) exit rates, (6) time spend onan individual web page, and/or (7) other competitive data found online.

Embodiments described herein may design a price optimization test usingtwo groups of products, a control group and a test group. The controlgroup and test group may include products that are the same make andmodel with similar attributes that have alike online sales. However, thecontrol group and test group may include products with a variable thatis different, such as color. For example, in one embodiment a controlgroup may include cherry patio furniture and the test group may includepine patio furniture of the same make and model of the control group.

The price test may cover different price points for products in the testgroup across multiple channels to optimize everyday pricing, seasonalpromotions, inventory clearance, freight pricing and cross sell/up selldata. Determining a price test may include receiving baseline businessrequirements or rules from a customer, such as a product minimum margin,price relationship, etc. Embodiments may be implemented in aSoftware-As-A-Service (SAAS) environment. The following provides a setof example factors that may be considered in a price test: (1) price vs.volume, (2) competitors' response to price changes, (3) product costshipping cost, handling cost, etc., (4) inventory level, inventoryreplenishment, (5) seasonality, (6) product lifecycle, (7) brandrecognition—both product brand and company brand, and/or (8) pricingstrategy.

Price testing may then be performed on the control group and the testgroup. The price testing may then analyze to determine how the productsin the control group and the test group are priced against associatedproducts being sold and/or carried by relevant competitors.

The price optimization system may then modify the price of products inthe test group, while maintaining the price of the products in thecontrol group, and analyze competitor's price change response toproducts that are matches to the products in the test group. Embodimentsdisclosed herein can also leverage a novel approach disclosed inAppendix D attached herewith to compare and/or analyze a competitor'sproducts with the products in the test group, and Appendix D is herebyincorporated in its entirety by reference.

The price optimization system may then analyze the test group todetermine if the prices of the products within the test group should bemodified and what testing strategy should be implemented on the testgroup. Utilizing the competitive data, the price optimization system mayalso determine if it makes sense to modify the price of a product for atest based on the competitor's pricing of the product. In oneembodiment, if the consumer's products price is already the lowest pricefor a product it might not make sense to further lower the price of theproduct.

While a price test is running, the price optimization system maydetermine a competitor's responses to price changes of the same productand changes to consumer behavior on the channel in response to thechange of pricing data for the test group. The price optimization systemmay also determine if competitors have modified their pricing ofproducts and how quickly they have changed the pricing of their productsas a result of the prices of the products within the test group beingmodified.

The price optimization system may compile web analytic data associatedwith a consumers interactions with the customer's channels such as wherea consumer is viewing the channel from, did the consumer click throughthe channel from Google shopping, directly, from a Google Adwordscampaign, etc., conversion rate, what website the consumer exited thechannel, how much time a consumer spent on the product page, etc.Therefore, while a price test is running, not only may sales be tracked,but additional data associated with the behavior of consumers atdifferent channels may be compiled and stored.

After performing a price sensitivity analysis, data may be compiledbased on the web analytics, conversions rates, revenue, etc. between thetest group and the control group, changes to the competitors' pricing ofthe products, and/or web analytics of the various channels to determinean optimized pricing of the customer's products. Price recommendationscan then be prepared and delivered or otherwise made available to thecustomer of the price optimization system. Embodiments create value tothe customer through increased revenue and/or profit, and may also be asource of increased revenue for product offering.

These, and other, aspects will be better appreciated and understood whenconsidered in conjunction with the following description and theaccompanying drawings. The following description, while indicatingvarious embodiments and numerous specific details thereof, is given byway of illustration and not of limitation. Many substitutions,modifications, additions or rearrangements may be made within the scopeof this disclosure, which includes all such substitutions,modifications, additions or rearrangements.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings accompanying and forming part of this specification areincluded to depict certain aspects of various embodiments. A clearerimpression of these embodiments, and of the components and operation ofsystems provided with them, will become more readily apparent byreferring to the exemplary, and therefore nonlimiting, embodimentsillustrated in the drawings, wherein identical reference numeralsdesignate the same components. Note that the features illustrated in thedrawings are not necessarily drawn to scale.

FIG. 1 depicts a block diagram of one embodiment of an architecture inwhich a price sensitivity and optimization system may be implemented.

FIG. 2 depicts a flow chart illustrating an example operation of oneembodiment of a price optimization system disclosed herein.

FIG. 3 depicts a flow chart illustrating an example operation ofdesigning a price test.

FIG. 4 depicts a flow chart illustrating an example price elasticitycurve.

FIG. 5 depicts a table illustrating different type of tests according toone embodiment disclosed herein.

FIGS. 6-8 depict screenshots of example price tests according toembodiments disclosed herein.

DETAILED DESCRIPTION

Various features and advantageous the present disclosure are explainedmore fully with reference to the nonlimiting embodiments that areillustrated in the accompanying drawings and detailed in the followingdescription. Descriptions of well-known starting materials, processingtechniques, components and equipment are omitted so as not tounnecessarily obscure the present disclosure. It should be understood,however, that the detailed description and the specific examples, whileindicating preferred embodiments, are given by way of illustration onlyand not by way of limitation. Various substitutions, modifications,additions and/or rearrangements within the spirit and/or scope of theunderlying inventive concept will become apparent to those skilled inthe art from this disclosure. Embodiments discussed herein can beimplemented in suitable computer-executable instructions that may resideon a computer readable medium (e.g., a hard disk (HD)), hardwarecircuitry or the like, or any combination.

Before discussing specific embodiments, a brief overview of the contextof the disclosure may be helpful. In this disclosure, the term“customer” may refer to a customer of the pricing system, “consumer” mayrefer to an end user of a customer's online shopping engine, “channel”may refer to a virtual or physical avenue with which a customer promotestheir product(s), and “product” may refer to a customer's product soldthrough one of their channels. Example channels may include, but are notlimited to, Google Adwords, Google shopping, brick and mortar stores,websites, etc.

Systems and methods described herein enable customers to determine pricesensitivity, price elasticity, and price optimization for productsegments for cross-selling, up-selling, seasonal promotions, inventoryclearance, freight pricing, etc. Embodiments may include a priceoptimization system configured to design price tests, run the tests,monitor the tests, adjust the tests if desired, and analyze the resultsof the tests to determine the test's significance, price sensitivity ofthe product segment, and/or price optimization of the product segment.The results of the tests may be communicated to the customer and be usedto determine inputs of further tests for the product segment.

Turning now to FIG. 1, a block diagram illustrating an exemplary system100 is shown. System 100 couples to a network such as the Internet 101and has access to domains 110 a . . . 110 n. Each domain may have acommon network name (domain name) under which a collection of networkdevices are organized (e.g., domain.com). Each domain may have one ormore sub-domains (e.g., abc.domain.com, xyz.domain.com, etc.) accordingto the hierarchical Domain Name System (DNS) of the Internet. Thecollection of network devices in a domain may include one or more servermachines hosting a website representing the domain (e.g.,www.domain.com).

A website (also referred to as Web site, web site, or site) refers to aset of related web pages (also referred to as pages) containing contentsuch as text, images, video, audio, etc. A website can be accessible viaa network such as the Internet or a private local area network throughan Internet address known as a Uniform Resource Locator (URL). Allpublicly accessible websites collectively constitute the World Wide Web.

Crawler 130 of system 100 may crawl the Internet 101 across domains 110a-110 n for data and store them in raw data database 140. The dataobtained by crawler 130 may be associated with retail products. Wrappers160 may be generated using techniques disclosed in Appendix A and/orAppendix B to extract desired information, such as domain product andprice information, from the raw data obtained by crawler 130. Othersuitable wrapper generation techniques may also be used. The domainproduct and price information for competitors thus obtained may bestored at database 170.

System 100 may include competitive price optimizer 120. Component 120may include data gathering module 148, design test module 149, run testmodule 150, sensitivity analysis module 151, price recommendation module152 and interface module 154. Functionality of these features will nowbe described in detail.

Data gathering module 148 may be configured to gather data such ascustomer channel web analytics, competitor data associated withproducts, and customer business rules. In one embodiment, the customerwebsite analytics may be obtained via FTP for individual consumers onchannels at the product level. Competitor data may be acquired byquerying database 170. Customer business rules may be obtained from acustomer, and may indicate business rules associated with a product suchas lowest allowable margin, season products, etc. Furthermore, datagathering module 148 may identify the top selling products by channeland product category over a period of time, such as the past threemonths. For the top selling products, data gathering module 148 may beconfigured to obtain competitors data over the period of time for eachproduct, and it may be desired to design tests for the top sellingproducts.

Design test module 149 may be configured to define test and controlgroups. Design test module 149 may be configured to determine the testand control groups by selecting a group of products of similar category,sales volume, price, channel and type (i.e. patio chairs, wood in therange of $100-$125 with 500 purchases per month, on Google ShoppingChannel). Design test module 149 may then split products into a test andcontrol group based on similarity, where two products that are the mostsimilar based on defined metrics may become the test and control group.

Design test module 149 may be configured to define price points based oncompetitor data, channel, business rules, and market dynamics for theproducts in the test group, where only the test group products pricepoints are modified. Each product in the test group may be assignedmultiple price points during an executed test.

In one embodiment, if a current price point for a product is too low,the price point for the product in the test group may be increasedincrementally. In one embodiment, break-even analysis and customerbusiness rules may be applied to define the price points. The pricepoints may be normalized for seasonality, market dynamics, and designchanges on the site/channel. Initially, at the start of a test, designtest module 149 may be configured to move up and down the price point ofthe products in the test groups in smaller increments and increase theincrements of the price points of the products in the test group basedon current and competitive position of the products in the product groupcompared to the price points of their competitors, their margin value,etc. Design test module 149 may also be configured to determine thelength of a test, which may be any desired length of time.

In one embodiment, design test module 149 may also determine tests basedon existing traffic volume of a customer's website, price variation,expected price change and a significance threshold associated with thechange of products price, wherein the significance threshold may beassociated with product revenue, margin, volume, sales, etc. Design testmodule 149 may also determine tests based on acceptable impact on thecustomer's business, the type of competition based on the brand equityof determined competitors, a number of competitors for a product group,and competitors response by channel.

Run test module 150 may be configured to execute a test created bydesign test module 149. Run test module 150 may be configured to delivera list of price changes per day for the products in test group, and acustomer may change the prices of the products in the test group. Inother embodiments, run test module 150 may change the prices of theproducts in the test group. Run test module 150 may measure customerdata associated with a margin, profit, price and volume of the productsin the test group and the control group during an executed test. Runtest module 150 may also measure changes to competitor's products thatare associated with the test group such as the brand equity of thecompetitor, how the competitor responds to the change in price such ashow often and by how much has the competitor changed the price of theproducts associated with the test group, and the price rank of theproducts in the test group based on competitor's responses to the pricepoints.

Run test module 150 may also measure web analytics of the customer ondifferent channels, such as the number of unique visits, conversionrates, exit rates, time spent on the webpage, time to first purchase,etc. Run test module 150 may also be configured to intervene with a testif competitor responses are different than what was anticipated, andalter a test mid-test based on the measurements.

Sensitivity analysis module 151 may be configured to test thesignificance of each price point in the test using any statisticalanalysis test, such as the chi-squared test. Sensitivity analysis module151 may also be configured to analyze correlations between individualcompetitor responses and their effects on web analytics for thecustomer's channels. In one embodiment, sensitivity analysis module 151may be configured to generate an ensemble model based on anycorrelations and price/volume data from the test. Using regressionanalysis sensitivity analysis module 151 may calculate the elasticity ofprice based on the generated ensemble model.

Price recommendations module 152 may be configured to recommend a pricefor products based on the price sensitivity to optimize revenue, profit,sales, etc. The price for the products may be communicated to a customervia any known mechanism such as a data feed. Price recommendationsmodule 152 may continuously monitor the metrics for design test module149 and correspondingly update the sensitivity calculations anddetermine an optimized price for corresponding products.

As describe above, a customer of system 100 can interact with system 100via a user interface provided by interface module 154. Inputs providedby the customer at the front end (e.g., via a web browser running on aclient device associated with the customer and implementing an instanceof a web based user interface provided by interface module 154) may becommunicated to a server machine running system 100 (or a portionthereof, e.g., component 120) at the back end and stored in a data store(not shown) accessible by design test module 149, run test module 150,sensitivity module 151 and price recommendation module 152.

FIG. 2 depicts a flow chart illustrating an example operation of oneembodiment of a price optimization system. At the data gathering step210, customer analytics, competitor data, and customer business rulesmay be gathered. Customer analytics refers to web analytics for acustomer's site. Web analytics refers to the measurement, collection,analysis and reporting of internet data for purposes of understandingand optimizing web usage. In this case, the customer site web analyticsmay provide data on consumer behavior and/or interactions with thecustomer's site concerning the customer's products. For example, thecustomer site web analytics can provide site analytics data pertainingto an individual consumer per channel at the product level. In oneembodiment, the customer website analytics may be obtained from thecustomer via FTP. In one embodiment, competitor data may be obtainedfrom database 170. For example, competitor data may be obtained byquerying an items table stored in database 170. In one embodiment,business rules may be provided by the customer via an interfacegenerated by interface module 154. Those skilled in the art willappreciate that such an interface can be implemented in various ways.For example, interface module 154 may provide an interview- orwizard-style interface in which an authorized user for the customeranswers questions presented to the user and the answers provided by theuser are used to formulate business rules. Example questions mayinclude, but are not limited to, “What is the lowest allowable margin?”,“Is this a seasonal product?”, etc.

In one embodiment, the top selling products by channel and category fora particular time period are identified. In one embodiment, the topselling products by channel and category for the past three months maybe identified. For these tope selling products, the competitor data forthe past three months, including all competitors for each product alongwith all competitor product data associated therewith, are obtained fromthe items table stored in database 170.

At the test designing step 220, the customer may design a test via auser interface provided by interface module 154. In one embodiment, indesigning a test, an authorized user for the customer may, via the userinterface, define a test group and a control group, define price pointsto test, and define the length of the test. As an example, the systemmay work with the customer to:

Define Test and Control Groups: (1) Of the top selling products andpreferred categories, select a group of 10 products of similar category,sales volume, price, channel and type. For example, a customer mayselect patio chairs that are made of wood, in the range of

$100-$125, have at least 500 purchases per month, and the primarychannel being Google Shopping. (2) Split these products into two groupsbased on similarity: Products that are most similar based on definedmetrics are placed in the test and control groups.

Define Test and Control Groups: (1) Each product in the test group canbe assigned multiple price points to test based on competitor data andchannel. For example, if the current price point is low, the system canmove the price up incrementally. See FIG. 5 for example test types. (2)Apply break-even analysis and customer business rules. (3) Normalize forseasonality, market dynamics, and design changes on the customer's site.(4) Price points can move up and down in small increments at thebeginning of the test and grow in size toward the end of the test basedon current and expected competitive position. For example, price pointscan move up and down in approximately $0.10 increments at the beginningof the test and grow in size to approximately $1.00 increments towardthe end of the test.

Define length of test: (1) Determine existing traffic volume, variation,expected change and significance threshold. In one embodiment, thesequestions can be answered after the first test run. (2) Determineacceptable impact on business, the type of competition, competitors'responses by channel, etc. The length of a test may depend on the numberof big players versus small players, the number of competitors in thesame product category and/or channel, etc.

An example operation of designing a price test will be further describedbelow with reference to FIG. 3. At the test running step 230, the systemmay operate to run the test for the customer. To begin the test, a listof price changes per day for the products in the test group may bedelivered to the customer. The customer may, via the above-describedinterface, change the prices of the products in the test group as perthe test design. While the test is being run, the following measurementsmay be taken: (1) Customer: Margin and profit (price vs. volume), (2)Competition: Brand equity (unique visits), responses (how often, by howmuch), price rank, (3) Customer: Unique visits, conversion rate, exitrate, time spent on page, time to first purchase.

For example, the system may operate to measure the customer's margin andprofit on the products, as well as changes to competitor's products thatare associated with the test group such as the brand equity of thecompetitor, how a competitor responds to the change in price such as howoften and by how much has the competitor changed the price of theproducts associated with the test group, and the price rank of theproducts in the test group. Also, while the test is being run, webanalytics of the customer's channels may be measured. Since competitorresponses may be different than anticipated, the test design may bealtered mid-test based on measurements taken thus far.

At the sensitivity analysis step 240, the following analyses may beperformed: (1) Significance Test—Test the statistical significance ofvolume change due to price change in the test using a statisticalanalysis, (2) Competitor Analysis—Test correlations between individualcompetitor responses and their effects on the customer site webanalytics and create an ensemble model based on the correlations andprice/volume data from the test, (3) Sensitivity Analysis—Calculate theelasticity of price base on the ensemble model using a regressionanalysis.

For example, the significance of volume change in the test may bedetermined using the chi-squared test. The sensitivity of the test mayalso be analyzed to measure correlations between individual competitorresponses and their effects on unique visits, conversion rate, exitrate, time spent on page, time to first purchase, etc. with respect tothe customer's channel(s). An ensemble model can be created based onthese correlations and price/volume data from the test. Applying aregression analysis on the model, the elasticity of the price for theproducts may be calculated.

As described above, while designing a test, potential competitor'sresponses may be considered depending on the channel and the velocity oftraffic on the channel. The designed test may also be competed onindividual channels for price differentiation.

The above-described steps (design a test, run the test, and analyze thetest) may be repeated per category (step 245). The results of each testcan be analyzed to determine modifications to consumer's behaviors basedon the price points of the products being modified during the pricetest. The results of the test can also be analyzed to determinerecommendations for the customer.

At the price recommendations step 250, once price sensitivity iscalculated (which, in one embodiment, produces a number), it can be usedto optimize for revenue or profit. In one embodiment, an IBM ILOG CPLEXOptimization software package (also referred to as CPLEX Optimizer) canbe used to optimize for revenue or profit based on the calculated pricesensitivity. Other optimizers may also be used. Price recommendations tooptimize revenue or profit based on the price sensitivity can bedelivered or otherwise communicated to the customer in various ways. Forexample, the system may deliver price recommendations to the customervia a data feed.

The system may, in one embodiment, continuously monitor the metricsoutlined in the testing phase described above and update sensitivitycalculations and optimized pnce. To this end, price recommendations forthe products involved in the test may be continuously monitored,updated, and communicated to the customer.

The following are non-exclusive factors that may be analyzed aftercompleting a test: (1) Price vs. Volume changes, (2) How many keycompetitors' in the market (at product level)?, (3) Who is the priceleader and do other competitors follow?, (4) How often do my competitorschange their prices?, (5) Where is a customer's price position now andwhere should the customer's price position be?, (6) How would mycompetitors respond to the customer's move?, (7) How much additionalvolume can the customer gain/lose by changing prices?.

The following is a nonexclusive list of consumer behavior that may beanalyzed: (1) # of unique visitors/# of visits, (2) Conversion rate, (3)Exit rate, (4) Time spent on individual page, (5) Time for first visitto purchase.

FIG. 3 depicts a flow chart illustrating an example operation ofdesigning a price test. At step 310, a price test may be designed forselected products based on competitor data, channel, business rules, andmarket dynamics. In one embodiment, if a current price point for aproduct is too low, the price point for the product may be increasedincrementally. The price points may be normalized for seasonality,market dynamics, and design changes on the site/channel. The designedtest may include multiple price points for the products in a productgroup. This allows for the building of a price elasticity curve, anexample of which is depicted in FIG. 4, so that the optimal price forrevenue, profit, margin, etc. can be determined. The designed test mayalso be completed over a given length of time as determined by empiricalevidence in order to achieve statistical significance.

At steps 320 and 330, a test group and a control group for the selectedproducts that are of similar category, sales volume, price, channel andtype may be defined. Each product in the test group may be assignedmultiple price points to test. Only the test group products price pointsare modified. Since only the products in the test group receive priceupdates, the impact of other factors outside of price changes such asseasonality can be reduced.

At step 340, the significance of the test, the sensitivity of thepricing of the products, and price optimization of the products may bedetermined based on competitors' responses to the price changes, webanalytics of the customer's channels, and market analysis associatedwith the products corresponding to the designed test. At step 350,additional tests may be designed for the product category, or new testsmay be designed for other product categories.

FIG. 4 depicts an example price elasticity curve 400 showing differenttested price points and how the price points affect the profit for aproduct category. More specifically, price elasticity curve 400 showsthat, as the price of a product increases the profit associated with theproduct also increases to a certain point. After the certain point,profits are not increased by increasing the price of the product. Inthis example, three different price tests are designed, run, andanalyzed which, as shown in price elasticity curve 400, may result inpossible price recommendations that may optimize the current profit fora product category for a customer.

FIG. 5 depicts a table illustrating different type of tests according toone embodiment. As depicted in FIG. 5, tests may be designed for variouspurposes. For example, tests may be designed so as to increase a marginpercentage, to increase a discount percentage, to increase a pricespread, etc. In one embodiment, different testing strategies may be morebeneficial to different types of product categories and may result indifferent testing benefits. Example price tests designed based ondifferent strategies are depicted in FIGS. 6-8.

FIG. 6 depicts an example testing strategy 600 to move up the priceposition of a product. Products that fit this testing strategy mayinclude those having a price position low on the market and a highmargin which allows more room to move the prices. The current revenue ofthe product could be high or low. If the current revenue of the productis low, the testing of the product will be of low risk to the customer.If the current revenue of the product is high, then it may be desired toanalyze channels of the sources of revenue. Some channels may be moresensitive to price changes than others. In this example, a customer“Compact Appliance” carries a product “Panasonic MC-UL915 JetSpinCyclone—Vacuum cleaner—upright—bagless—red metallic” which has a priceposition low on the market as compared to the same product carried bytheir competitors and which has a high margin to allow for possibleprice adjustments. In one embodiment, a test may be designed to increasethe discount percentage. The price optimization system described abovemay operate to run the test and measure the incremental volume gaincaused by the increase in the discount percentage. The results from thetest can be used in the sensitivity analysis as described above.

FIG. 7 depicts an example testing strategy 700 to move down the priceposition of a product. Products that fit this testing strategy mayinclude those having a price position high on the market and a lowmargin in which a small increase in price may not affect the sale volumeof the product but may dramatically improve the margin. In this case,the current revenue of the product should be high. If the currentrevenue of the product is low, it may not justify the effort to performan analysis of the product since a price increase will likely lead toeven lower revenue. Furthermore, low revenue products may take a longtime to test. For high revenue products, the testing may pose aninsignificant risk to the customer, although the product should beclosely monitored for any change to the sale volume subsequent to eachsmall increase in price. In this example, a customer “Compact Appliance”wants to improve the profit margin on a product “LG LW1210HR 12,000 BTU208/230V Cool/Electric Heat” which has a price position high on themarket as compared to the same product carried by their competitors. Inone embodiment, a test may be designed to incrementally increase theprice of the product. The price increment may vary along the length ofthe test. As an example, the price increment may be relatively small,for instance, about 10 cents, at the beginning of the test. The priceincrement may be increased to about a dollar per an increase toward theend of the best. The price optimization system described above mayoperate to run the test and measure any increase in the margin and anyvolume loss subsequent to each increase in the price. The results fromthe test can be used in the sensitivity analysis as described above.

FIG. 8 depicts an example testing strategy 800 to optimize the pricedistribution and price position of a product on the market. Productsthat fit this testing strategy may include in-house products andproducts with dominating positions through multiple sites. Such productscould have a high or low profit margin. The current revenue of theproducts to be tested should be high as low revenue products have smallpotential for growth. With this strategy, price differentiation may beoptimized among sites and channels, and the value of brand names may bedetermined. For example, the same product could be sold by the samecustomer through different channels or sites under different brandnames. With this strategy, the product could be priced, tested, andoptimized under different brand names for different sites.

Additional price testing strategies may also be possible. For example,in some embodiments, tests may be designed for seasonal promotions wherea promotion schema appropriate for market segmentation may generate ahigher profit. In some embodiments, tests may be designed for inventoryclearance. Without a proper price test, inventory clearance prices couldbe set too low, leaving revenue under-optimized. In some embodiments,tests may be designed for freight pricing. Customers may offer freeshipping. However, a price test may reveal that a reasonable shippingfee may significantly increase the overall profit for large and/orexpensive items. In some embodiments, tests may be designed to comparedifferent pricing strategies such as the base price vs. the total price.For example, a lower base price may result in a better price position onshopping sites and thus more traffic, where the total price may ensure adesired margin. In some embodiments, tests may be designed to analyzecross selling and/or up selling. Such a test may be designed at thecustomer level rather than at the product level. At the customer level,valuable information about consumers may be obtained. Examples ofconsumer information may include which products consumers browse andwhich channels they visit, what is common across similar shopping cartsor baskets, what price ranges of these products, what otherproducts/websites that they consumers visit (which may be obtained viacookies), etc.

Although the present disclosure has been described in terms of specificembodiments, these embodiments are merely illustrative, and notrestrictive. The description herein of illustrated embodiments,including the description in the Abstract and Summary, is not intendedto be exhaustive or to limit the disclosure to the precise formsdisclosed herein (and in particular, the inclusion of any particularembodiment, feature or function within the Abstract or Summary is notintended to limit the scope of the disclosure to such embodiments,features or functions). Rather, the description is intended to describeillustrative embodiments, features and functions in order to provide aperson of ordinary skill in the art context to understand the presentdisclosure without limiting same to any particularly describedembodiment, feature or function, including any such embodiment featureor function described in the Abstract or Summary. While specificembodiments are described herein for illustrative purposes only, variousequivalent modifications are possible, as those skilled in the relevantart will recognize and appreciate. As indicated, these modifications maybe made in light of the foregoing description of illustrated embodimentsand are to be included within the spirit and scope of the disclosure.Thus, various changes and substitutions are intended in the foregoingdisclosures, and it will be appreciated that in some instances somefeatures of embodiments will be employed without a corresponding use ofother features without departing from the scope and spirit as set forth.Therefore, many modifications may be made to adapt a particularsituation or material.

Reference throughout this specification to “one embodiment,” “anembodiment,” or “a specific embodiment” or similar terminology meansthat a particular feature, structure, or characteristic described inconnection with the embodiment is included in at least one embodimentand may not necessarily be present in all embodiments. Thus, respectiveappearances of the phrases “in one embodiment,” “in an embodiment,” or“in a specific embodiment” or similar terminology in various placesthroughout this specification are not necessarily referring to the sameembodiment. Furthermore, the particular features, structures, orcharacteristics of any particular embodiment may be combined in anysuitable manner with one or more other embodiments. It is to beunderstood that other variations and modifications of the embodimentsdescribed and illustrated herein are possible in light of the teachingsherein.

In the description herein, numerous specific details are provided, suchas examples of components and/or methods, to provide a thoroughunderstanding of described embodiments. One skilled in the relevant artwill recognize, however, that an embodiment may be able to be practicedwithout one or more of the specific details, or with other apparatus,systems, assemblies, methods, components, materials, parts, and/or thelike. In other instances, well-known structures, components, systems,materials, or operations are not specifically shown or described indetail to avoid obscuring aspects of embodiments. A person of ordinaryskill in the art will recognize that additional embodiments are readilyunderstandable from the disclosure.

Embodiments discussed herein can be implemented in a computercommunicatively coupled to a network (for example, the Internet),another computer, or in a standalone computer. As is known to thoseskilled in the art, a suitable computer can include a central processingunit (“CPU”), at least one read-only memory (“ROM”), at least one randomaccess memory (“RAM”), at least one hard drive (“HD”), and one or moreinput/output (“I/O”) device(s). The I/O devices can include a keyboard,monitor, printer, electronic pointing device (for example, mouse,trackball, stylist, touch pad, etc.), or the like.

ROM, RAM, and HD are computer memories for storing computer-executableinstructions executable by the CPU or capable of being complied orinterpreted to be executable by the CPU. Suitable computer-executableinstructions may reside on a computer readable medium (e.g., ROM, RAM,and/or HD), hardware circuitry or the like, or any combination thereof.Within this disclosure, the term “computer readable medium” or is notlimited to ROM, RAM, and HD and can include any type of data storagemedium that can be read by a processor. For example, a computer-readablemedium may refer to a data cartridge, a data backup magnetic tape, afloppy diskette, a flash memory drive, an optical data storage drive, aCD-ROM, ROM, RAM, HD, or the like. The processes described herein may beimplemented in suitable computer-executable instructions that may resideon a computer readable medium (for example, a disk, CD-ROM, a memory,etc.). Alternatively, the computer-executable instructions may be storedas software code components on a direct access storage device array,magnetic tape, floppy diskette, optical storage device, or otherappropriate computer-readable medium or storage device.

Any suitable programming language can be used, individually or inconjunction with another programming language, to implement theroutines, methods or programs of embodiments described herein, includingC, C++, Java, JavaScript, HTML, or any other programming or scriptinglanguage, etc. Other software/hardware/network architectures may beused. For example, the functions of the disclosed embodiments may beimplemented on one computer or shared/distributed among two or morecomputers in or across a network. Communications between computersimplementing embodiments can be accomplished using any electronic,optical, radio frequency signals, or other suitable methods and tools ofcommunication in compliance with known network protocols.

Different programming techniques can be employed such as procedural orobject oriented. Any particular routine can execute on a single computerprocessing device or multiple computer processing devices, a singlecomputer processor or multiple computer processors. Data may be storedin a single storage medium or distributed through multiple storagemediums, and may reside in a single database or multiple databases (orother data storage techniques). Although the steps, operations, orcomputations may be presented in a specific order, this order may bechanged in different embodiments. In some embodiments, to the extentmultiple steps are shown as sequential in this specification, somecombination of such steps in alternative embodiments may be performed atthe same time. The sequence of operations described herein can beinterrupted, suspended, or otherwise controlled by another process, suchas an operating system, kernel, etc. The routines can operate in anoperating system environment or as stand-alone routines. Functions,routines, methods, steps and operations described herein can beperformed in hardware, software, firmware or any combination thereof.

Embodiments described herein can be implemented in the form of controllogic in software or hardware or a combination of both. The controllogic may be stored in an information storage medium, such as acomputer-readable medium, as a plurality of instructions adapted todirect an information processing device to perform a set of stepsdisclosed in the various embodiments. Based on the disclosure andteachings provided herein, a person of ordinary skill in the art willappreciate other ways and/or methods to implement the describedembodiments.

It is also within the spirit and scope of the disclosure to implement insoftware programming or code an of the steps, operations, methods,routines or portions thereof described herein, where such softwareprogramming or code can be stored in a computer-readable medium and canbe operated on by a processor to permit a computer to perform any of thesteps, operations, methods, routines or portions thereof describedherein. Various embodiments may be implemented by using softwareprogramming or code in one or more general purpose digital computers, byusing application specific integrated circuits, programmable logicdevices, field programmable gate arrays, optical, chemical, biological,quantum or nanoengineered systems, or components and mechanisms may beused. In general, the functions of various embodiments can be achievedby any means as is known in the art. For example, distributed, ornetworked systems, components and circuits can be used. In anotherexample, communication or transfer (or otherwise moving from one placeto another) of data may be wired, wireless, or by any other means.

A “computer-readable medium” may be any medium that can contain, store,communicate, propagate, or transport the program for use by or inconnection with the instruction execution system, apparatus, system ordevice. The computer readable medium can be, by way of example only butnot by limitation, an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system, apparatus, system, device,propagation medium, or computer memory. Such computer-readable mediumshall generally be machine readable and include software programming orcode that can be human readable (e.g., source code) or machine readable(e.g., object code). Examples of non-transitory computer-readable mediacan include random access memories, read-only memories, hard drives,data cartridges, magnetic tapes, floppy diskettes, flash memory drives,optical data storage devices, compact-disc read-only memories, and otherappropriate computer memories and data storage devices. In anillustrative embodiment, some or all of the software components mayreside on a single server computer or on any combination of separateserver computers. As one skilled in the art can appreciate, a computerprogram product implementing an embodiment disclosed herein may compriseone or more non-transitory computer readable media storing computerinstructions translatable by one or more processors in a computingenvironment.

A “processor” includes any, hardware system, mechanism or component thatprocesses data, signals or other information. A processor can include asystem with a general-purpose central processing unit, multipleprocessing units, dedicated circuitry for achieving functionality, orother systems. Processing need not be limited to a geographic location,or have temporal limitations. For example, a processor can perform itsfunctions in “real-time,” “offline,” in a “batch mode,” etc. Portions ofprocessing can be performed at different times and at differentlocations, by different (or the same) processing systems.

It will also be appreciated that one or more of the elements depicted inthe drawings/figures can also be implemented in a more separated orintegrated manner, or even removed or rendered as inoperable in certaincases, as is useful in accordance with a particular application.Additionally, any signal arrows in the drawings/figures should beconsidered only as exemplary, and not limiting, unless otherwisespecifically noted.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having,” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,product, article, or apparatus that comprises a list of elements is notnecessarily limited only those elements but may include other elementsnot expressly listed or inherent to such process, process, article, orapparatus.

Furthermore, the term “or” as used herein is generally intended to mean“and/or” unless otherwise indicated. For example, a condition A or B issatisfied by any one of the following: A is true (or present) and B isfalse (or not present), A is false (or not present) and B is true (orpresent), and both A and B are true (or present). As used herein,including the claims that follow, a term preceded by “a” or “an” (and“the” when antecedent basis is “a” or “an”) includes both singular andplural of such term, unless clearly indicated within the claim otherwise(i.e., that the reference “a” or “an” clearly indicates only thesingular or only the plural). Also, as used in the description hereinand throughout the claims that follow, the meaning of “in” includes “in”and “on” unless the context clearly dictates otherwise.

What is claimed is:
 1. A computer-implemented method comprising:generating a control set, using a processor of a system that is coupledto the internet, based on first input received via a user interface, thecontrol set comprising at least a first product of a product categorytype; generating a test set, using the processor, based on second inputreceived via the user interface, the test set comprising at least asecond product of the product category type other than the first productin the control set; changing, using the processor, a feature of thesecond product in the test set, the feature being visible on a web pagevia the internet, while maintaining the feature of the first product inthe control set, wherein the feature comprises price, and wherein thechanging comprises making multiple price changes of differentincrements; measuring, using the processor, competitor or consumerresponses to the changing of the feature of the second product in thetest set, wherein the measuring the competitor or consumer responses tothe changing comprises obtaining competitor data from one or more websites of competitors by: crawling the internet to obtain raw competitordata from one or more domains associated with the one or more websitesof competitors and applying one or more wrappers to the raw competitordata to automatically extract competitor responses from the rawcompetitor data; and generating a recommendation, using the processor,based on the measured competitor or consumer responses.
 2. The method ofclaim 1, wherein the different increments comprises at least one laterincrement that is larger than an earlier increment.
 3. The method ofclaim 1, wherein the measuring the competitor or consumer responses tothe changing comprises collecting internet data comprising at least oneof unique visitors to the web page, visitor time spent on the web page,or visitor time from first visit to the web page to purchase of thesecond product in the test set.
 4. The method of claim 3, wherein thegenerating of the recommendation is further based on the collectedinternet data.
 5. The method of claim 1, further comprisingimplementing, using the processor, the recommendation such that anadjustment to the feature of the second product of the test setaccording to the recommendation appears on the web page.
 6. The methodof claim 1, the measuring of the competitor or consumer responses to thechanging comprises at least one of measuring a change in price of atleast one competitor product or measuring how quickly the price of theat least one competitor product changed.
 7. The method of claim 1, themeasuring the competitor or consumer responses comprises: comparinganalytic information with respect to the control set to analyticinformation with respect to the test set, wherein the analyticinformation comprises at least one of: consumer click throughinformation, and conversion rate, and wherein the recommendation isfurther generated based on the comparison.
 8. The method of claim 1,wherein the measuring the competitor or consumer responses to thechanging comprises collecting, using the processor, customer datacomprising behavior of customers in at least one physical store locationrelating to the second product in the test set.
 9. A non-transitorycomputer readable medium having instructions stored thereon that, uponexecution by a computing device, cause the computing device to: generatea control set, using a processor of a system that is coupled to theinternet, based on first input received via a user interface, thecontrol set comprising at least a first product of a product categorytype; generate a test set, using the processor, based on second inputreceived via the user interface, the test set comprising at least asecond product of the product category type other than the first productin the control set; change, using the processor, a feature of the secondproduct in the test set, the feature being visible on a web page via theinternet, while maintaining the feature of the first product in thecontrol set, wherein the feature comprises price, and wherein thechanging comprises making multiple price changes; measure, using theprocessor, competitor or consumer responses to the change of the featureof the second product in the test set, wherein the measurement of thecompetitor or consumer responses to the change comprises obtainingcompetitor data from one or more web sites of competitors such that theinstructions further cause the computing device to: crawl the internetto obtain raw competitor data from one or more domains associated withthe one or more websites of competitors and apply one or more wrappersto the raw competitor data to automatically extract competitor responsesfrom the raw competitor data; and generate a recommendation, using theprocessor, based on the measured competitor or consumer responses. 10.The non-transitory computer readable medium of claim 9, wherein therecommendation is a recommendation to change at least one of a price ofthe first product in the control set or a price of the second product inthe test set.
 11. The non-transitory computer readable medium of claim9, wherein the recommendation is a recommendation to change a price of athird product of the product category type.
 12. The non-transitorycomputer readable medium of claim 9, where in the recommendation is arecommendation to change a price of the second product in the test setwith respect to a particular channel in which the second product issold.
 13. A system comprising: a memory; and a processor coupled to thememory, wherein the processor is configured to: generate a control set,using a processor of a system that is coupled to the internet, based onfirst input received via a user interface, the control set comprising atleast a first product of a product category type; generate a test set,using the processor, based on second input received via the userinterface, the test set comprising at least a second product of theproduct category type other than the first product in the control set;change, using the processor, a feature of the second product in the testset, the feature being visible on a web page via the internet, whilemaintaining the feature of the first product in the control set, whereinthe feature comprises price, and wherein the changing comprises makingmultiple price changes; measure, using the processor, competitor orconsumer responses to the change of the feature of the second product inthe test set, wherein the measurement of the competitor or consumerresponses to the change comprises obtaining competitor data from one ormore web sites of competitors such that the processor is furtherconfigured to: crawl the internet to obtain raw competitor data from oneor more domains associated with the one or more websites of competitorsand apply one or more wrappers to the raw competitor data toautomatically extract competitor responses from the raw competitor data;and generate a recommendation, using the processor, based on themeasured competitor or consumer responses.
 14. The system of claim 13,wherein the user interface is displayed to a user via a web browserrunning on a client device.
 15. The system of claim 13, wherein themeasured competitor or consumer responses comprise information relatingto at least one of: what channel a consumer is viewing a second productpage through; click through information; at what web site or web pagethe consumer exited the channel; an exit rate of a plurality ofconsumers; a conversion rate of the plurality of consumers; time fromfirst visit of the consumer to the second product page to purchase ofthe second product; or how much time the consumers spend viewing thesecond product page.
 16. The system of claim 13, wherein the first inputspecifies the control set by indicating a category type of product, asales volume, a price, a specific channel, a type of channel, or anycombination thereof.
 17. The system of claim 13, wherein the processoris further configured to: perform a sensitivity analysis configured totest the significance of each price point tested during the change,wherein the recommendation is further generated based on results of thesensitivity analysis.